This repository hosts several SoC Benchmarks that have "system" vulnerability in them exploitable through application software (aka software exploitable vulnerabilities). It is inspired by similar benchmarks in hardware security: Trust-Hub and ISCAS for hardware trojans and logic locking respectively. Unlike the benchmark circuits provided in those databases, the system vulnerability examples provided in MITRE CVE and CWE databases are not "plug and play" in the sense that you cannot readily just download the examples and run your own test application on them. This is an impediment to security research as ideally one would want such convenience, to test whether dynamic verification methodologies are able to detect SoC vulnerabilities when it is executing a software workload. This repository/database aims to provide that convenience.
These vulnerable systems are meant to be representative "implementations" of the case that designer/integrator mistakes result in vulnerabilities described at a high-level in CWE/CVE databases. The benchmark SoCs are based on open source RISC-V SoCs, namely NEORV32 and Ibex. The datasheet available at the respecitve repositories are also applicable to these benchmarks as the individual IP compositions/designs and system wide memory map have not been tinkered with.
Each benchmark comes with its own C software execution environment which includes linker scripts, startup codes to handle exceptions and makefiles to compile appropriate executables. We also provide a pre-built bitstream file for each benchmark. Therefore, you can just write your own C code (or use the ones generated by your verification methodology), compile using the makefile provided, prototype the system on an FPGA and run said executable on the prototyped design. These benchmarks demonstrate some weakness/vulnerability in either the software or hardware layer that comporomise confidentiality, integrity and/or availability requirements of the security assets. Despite these weaknesses/vulnerabilities, they are "fully" functional insofar as you will be able to execute almost all valid C/assembly programs on these SoCs. "Functional" does not necessarily imply secure!
If you use any of our benchmarks in your research, kindly include the following citation in your work! Paper details to be included in later releases
The easiest way to get started with these benchmarks is to upload the pre-built bitstream to the corresponding Xilinx FPGA. You can find them in the fpga directory of corresponding benchmark folders. You can also find the required constraints file to synthesize these designs in these directories. Each directory also contains a demo blink LED program + exectuable which you can upload using OpenOCD + GNU GDB. The debugging section provides the instructions for doing so.
Each benchmark supports single-step debug of the executable running on it through OpenOCD and GNU GDB. There are 2 exectuables already provided in each benchmark folder which you can use or you can build your own binaries using the instructions available in a later section. You will require >v12.0 for RISC-V port of OpenOCD and the RISC-V GNU toolchain. Get them from here and here. To upload and debug the executable, first open up a terminal in the root directory of the benchmark you are using, and then execute openocd -f sw/openocd/the_cfg_file.cfg. In another terminal, invoke GDB using
riscv32-unknown-elf-gdb
and connect to the remote host using
target extended-remote localhost:3333
upload the executable using
file main.elf
load
Then you can set breakpoints and debug as required. More information/examples on single-step debugging is available here.
We would recommend using the prebuilt bitstreams since the purpose of these benchmarks is to promote gray/black box system verification methodologies. This means that you would treat the system as a kind of acquired 3PIP for which you do not have access to full implementation/microarchitectural knowledge. This is in keeping with the verification challenge faced by the industry and academia today (you can read more on these challenges through these excellent resources: 1, 2, 3). The software "exploit" that will make the security requirement violation apparent is present in exploit directory of corresponding benchmarks.
However, we also provide RTL of the system hardware present in each directory that you could utilize for more white box approaches (such as formal & UVM).
We realize that you may not possess the boards that we have provided FPGA bitstreams for at your disposal. To help you with setting up the project for synthesis, we are providing you video tutorials that you can follow along to set up your own projects. The only change will be in the constraints file that you use for your own board. We do not provide constraint files for other boards but you can find master constraints file on the internet. You would just have be careful in using only the resources available on your specific board.
You will need to create your own directory under the sw folder of the corresponding benchmark and copy over the makefile available in either exploit or demo_blink_led folders. You can write your code in either C or assembly. You can run any valid code provided that they do not require RISC-V extensions beyond what is present and/or dynamic memory allocation on the heap (a future release will allow dynamic memory allocation as well). You can find information on the extensions available by running i r misa on your GDB terminal once you have connected to the remote target. Once you have written your main code, execute make USER_FLAGS+=-g clean_all all to build the binary. If your code uses header files, keep them in lib/include directory or change corresponding settings in the common makefile available in sw/common directory. Once the binary is built, you can upload and debug following instructions provided in the debugging section.
This tutorial will show you the steps you need to follow to get your Vivado project up and running. It also shows the commands you would need to follow to debug an executable running on the SoC prototyped on the FPGA. Note that you may need to adjust certain steps (e.g., setting breakpoint no. in gdb) according to the program you are debugging/board you are using.
Video1 link
Video2 link
Each of the vulnerabilities present in the benchmarks have been based on our interpretation/implementation of MITRE CWEs. This table describes the vulnerabilities:
| Benchmarks | Based On | FPGA Support | Vulnerability Description | Sec. Requirement Violated | Reference |
|---|---|---|---|---|---|
| VSoCNRVH-1 | Neorv32 | Digilent Genesys 2 | U-mode applications can access and modify privileged M-mode CSRs | Integrity | CWE-1274, CWE-1256 |
| VSoCNRVS-1 | Neorv32 | Digilent Genesys 2 | Causing an exception from U-mode can lead to arbitrary code execution | Availability | CWE-1196, CVE-2022-20826 |
| VSoCNRVS-2 | Neorv32 | Digilent Genesys 2 | Trap handler does not handle exceptions properly | Availability | CWE-755, CWE-1332, CVE-2023-39801 |
| VSoCNRVH-2 | Neorv32 | Digilent Genesys 2 | U-mode applications can cause timeout when TW bit is set | Availability | CWE-1256 |
| VSoCNRVH-3 | Neorv32 | Digilent Genesys 2 | U-mode applications can read M-mode CSR counter | Confidentiality | CWE-1303, CVE-2022-42336 |
| VSoCIBSH-1 | Ibex | Digilent Nexys A7 | U-mode applications can access and modify privileged M-mode CSRs | Integrity | CWE-1274, CWE-1256 |
| VSoCIBSH-2 | Ibex | Digilent Nexys A7 | U-mode applications can cause timeout when TW bit is set | Availability | CWE-1256 |
| VSoCIBSS-1 | IBS | Digilent Nexys A7 | Trap handler does not handle exceptions properly | Availability | CWE-755, CWE-1332, CVE-2023-39801 |
Each benchmark is distributed under the BSD 3-clause license.